Question order and interviewer effects in CATI scale-up surveys

The scale-up estimator is a network-based estimator for the size of hidden or hard to count subpopulations. Several issues arise in the public health context when the aim is the estimation of injuries occurring in a certain population, where two common problems are present: (a) Small injuries are usually difficult to observe and rarely reported in the official data and (b) people are not always compliant in giving information about some specific injuries, in particular when children are involved. This study checked the methodological issues arising from using a computer-assisted telephone interview (CATI) survey using the scale-up methodology for detecting the number of injuries due to choking in children ages 0 to 14 in Italy. For this purpose, 1,000 CATI interviews were conducted during a week using a questionnaire based on 33 questions about populations of known size according to census data. Then, each respondent was asked about other questions related to the main target population (e.g., number of children Department of Statistics and Applied Mathematics, University of Torino, Italy S&A S.r.l. Surveys & Analyses, Torino, Italy Department of Public Health and Microbiology, University of Torino, Italy Laboratory of Epidemiological Methods and Biostatistics, Department of Environmental Medicine and Public Health, University of Padova Corresponding Author: Dario Gregori, MA, PhD, Laboratory of Epidemiological Methods and Biostatistics, Department of Environmental Medicine and Public Health University of Padova, 35121 Padova, Italy, Phone: +39 049 8215384, Fax: +39 02 700445089 E-mail: [email protected] Sociological Methods & Research 38(2) 287–305 a The Author(s) 2009 Reprints and permission: http://www. sagepub.com/journalsPermissions.nav DOI: 10.1177/0049124109346163 http://smr.sagepub.com at UNIV OF FLORIDA Smathers Libraries on December 1, 2009 http://smr.sagepub.com Downloaded from known to suffer from a choking accident). A sensitivity analysis was conducted for estimating the effect of varying subpopulations, order of the questions, and interviewer effects on the resulting estimates. For the interviewer effect, no particular differences were observed in the overall estimates of injuries. The conclusion is the scale-up estimator in association with CATI methodology shows a high potential in the field of injury prevention, being accurate and robust, but particular attention should be given to the training of the interviewers to improve stability of the estimates.